Towards a Software Product Line Assessment Framework

Empirical studies are important to evaluate new techniques, methodologies, tools and processes proposed. While the diversity of Software Product Line Engineering (SPLE) approaches is increasingly growing, empirical studies assessing them are still limited, thereby hampering the technology transfer of the field. One of the key reasons for this problem is the lack of support for designing studies or benchmarks to be an assessment basis to the SPLE community. This position paper proposes a framework for benchmarking SPLE efforts, guiding the identification and classification of candidate benchmark applications. Such applications should be used by the evaluation studies to assess new techniques, methodologies, tools and processes proposed. The framework defines attributes to classify and therefore evaluate, product line applications and their releases, which is demanded to also consider product line evolution.